OpenFed: A Comprehensive and Versatile Open-Source Federated Learning Framework

Recent developments in Artificial Intelligence techniques have enabled their successful application across a spectrum of commercial and industrial settings. However, these techniques require large volumes of data to be aggregated in a centralized manner, forestalling their applicability to scenarios wherein the data is sensitive or the cost of data transmission is prohibitive. Federated Learning alleviates these problems by decentralizing model training, thereby removing the need for data transfer and aggregation. To advance the adoption of Federated Learning, more research and development needs to be conducted to address some important open questions. In this work, we propose OpenFed, an open-source software framework for end-to-end Federated Learning. OpenFed reduces the barrier to entry for both researchers and downstream users of Federated Learning by the targeted removal of existing pain points. For researchers, OpenFed provides a framework wherein new methods can be easily implemented and fairly evaluated against an extensive suite of benchmarks. For downstream users, OpenFed allows Federated Learning to be plug and play within different subject-matter contexts, removing the need for deep expertise in Federated Learning.Comment: 18 pages, 3 figures, 1 tabl

Rethinking skip connection model as a learnable Markov chain

Over past few years afterward the birth of ResNet, skip connection has become the defacto standard for the design of modern architectures due to its widespread adoption, easy optimization and proven performance. Prior work has explained the effectiveness of the skip connection mechanism from different perspectives. In this work, we deep dive into the model's behaviors with skip connections which can be formulated as a learnable Markov chain. An efficient Markov chain is preferred as it always maps the input data to the target domain in a better way. However, while a model is explained as a Markov chain, it is not guaranteed to be optimized following an efficient Markov chain by existing SGD-based optimizers which are prone to get trapped in local optimal points. In order to towards a more efficient Markov chain, we propose a simple routine of penal connection to make any residual-like model become a learnable Markov chain. Aside from that, the penal connection can also be viewed as a particular model regularization and can be easily implemented with one line of code in the most popular deep learning frameworks~\footnote{Source code: \url{https://github.com/densechen/penal-connection}}. The encouraging experimental results in multi-modal translation and image recognition empirically confirm our conjecture of the learnable Markov chain view and demonstrate the superiority of the proposed penal connection.Comment: 12 pages, 4 figure

Deformable Object Tracking with Gated Fusion

The tracking-by-detection framework receives growing attentions through the integration with the Convolutional Neural Networks (CNNs). Existing tracking-by-detection based methods, however, fail to track objects with severe appearance variations. This is because the traditional convolutional operation is performed on fixed grids, and thus may not be able to find the correct response while the object is changing pose or under varying environmental conditions. In this paper, we propose a deformable convolution layer to enrich the target appearance representations in the tracking-by-detection framework. We aim to capture the target appearance variations via deformable convolution, which adaptively enhances its original features. In addition, we also propose a gated fusion scheme to control how the variations captured by the deformable convolution affect the original appearance. The enriched feature representation through deformable convolution facilitates the discrimination of the CNN classifier on the target object and background. Extensive experiments on the standard benchmarks show that the proposed tracker performs favorably against state-of-the-art methods

Aboveground Forest Biomass Estimation with Landsat and LiDAR Data and Uncertainty Analysis of the Estimates

Landsat Thematic mapper (TM) image has long been the dominate data source, and recently LiDAR has offered an important new structural data stream for forest biomass estimations. On the other hand, forest biomass uncertainty analysis research has only recently obtained sufficient attention due to the difficulty in collecting reference data. This paper provides a brief overview of current forest biomass estimation methods using both TM and LiDAR data. A case study is then presented that demonstrates the forest biomass estimation methods and uncertainty analysis. Results indicate that Landsat TM data can provide adequate biomass estimates for secondary succession but are not suitable for mature forest biomass estimates due to data saturation problems. LiDAR can overcome TM’s shortcoming providing better biomass estimation performance but has not been extensively applied in practice due to data availability constraints. The uncertainty analysis indicates that various sources affect the performance of forest biomass/carbon estimation. With that said, the clear dominate sources of uncertainty are the variation of input sample plot data and data saturation problem related to optical sensors. A possible solution to increasing the confidence in forest biomass estimates is to integrate the strengths of multisensor data

Operational Risk Aggregation across Business Lines Based on Frequency Dependence and Loss Dependence

In loss distribution approach (LDA), the most popular approach in operational risk modeling, frequency dependence and loss distribution dependence across business lines are two dependences which banks should consider. In practice, mainly for simplicity, many banks only model frequency dependence although they think that the impact of frequency dependence is insignificant. In this study, two approaches, respectively, models frequency dependence and loss distribution dependence, are introduced. Both approaches are modeled by copula function, which is capable of capturing nonlinear correlation. Based on the most comprehensive operational risk dataset of Chinese banking as far as we know, the operational risk capital charge of the overall Chinese banking is calculated by the two approaches. The results show that there is an obvious distinction between the capital calculated by modeling frequency dependence and the capital calculated by modeling loss dependence. The approach with very limited attention exactly yields a much larger capital result. So it is advised in this paper that banks should not just rely on the approach to modeling frequency dependence for it is natural and easy to deal with. A safer and more effective way for banks is to comprehensively take the results of the two kinds of approach into consideration

Operational Risk Aggregation across Business Lines Based on Frequency Dependence and Loss Dependence

In loss distribution approach (LDA), the most popular approach in operational risk modeling, frequency dependence and loss distribution dependence across business lines are two dependences which banks should consider. In practice, mainly for simplicity, many banks only model frequency dependence although they think that the impact of frequency dependence is insignificant. In this study, two approaches, respectively, models frequency dependence and loss distribution dependence, are introduced. Both approaches are modeled by copula function, which is capable of capturing nonlinear correlation. Based on the most comprehensive operational risk dataset of Chinese banking as far as we know, the operational risk capital charge of the overall Chinese banking is calculated by the two approaches. The results show that there is an obvious distinction between the capital calculated by modeling frequency dependence and the capital calculated by modeling loss dependence. The approach with very limited attention exactly yields a much larger capital result. So it is advised in this paper that banks should not just rely on the approach to modeling frequency dependence for it is natural and easy to deal with. A safer and more effective way for banks is to comprehensively take the results of the two kinds of approach into consideration